Introduction

In this Notebook, I have tried to build a model which predicts the age of a person based on his or her face.

The techniques I have used to build this Image Regression model are based on Fastai's CNN models.

The dataset I have used for this task are as follows:

  1. UTK Face Data
  2. Appa Real Face Data
  3. IMDB Wiki Face Data

In the course of this analysis, I have learnt many things:

  1. How to augment the data using Fastai's Image augmentation techinues.
  2. Loading and using Cadene's Pretrained Models (https://github.com/Cadene/pretrained-models.pytorch)
  3. How to use Image resizing technique which basically refers to gradually increase the size of image while training. This helps in achieving greater prediction accuracy.
  4. Discriminative Layers Training technique

Import Libraries

# This Python 3 environment comes with many helpful analytics libraries installed
# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python
# For example, here's several helpful packages to load in 

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)

# Input data files are available in the "../input/" directory.
# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory

import os
print(os.listdir("../input"))

# Any results you write to the current directory are saved as output.

import warnings
warnings.filterwarnings('ignore')
import gc
gc.collect()
['picture-2', 'picture4', 'wiki-face-data', 'picture-1', 'utk-face-cropped', 'picture-6', 'imdb-wiki-faces-dataset', 'picture-5', 'picture-3', 'appa-real-face-cropped']
0
!pip install pretrainedmodels

%reload_ext autoreload
%autoreload 2
%matplotlib inline

!pip install fastai==1.0.57
import fastai

from fastai import *
from fastai.vision import *

from torchvision.models import *
import pretrainedmodels

from utils import *
import sys

from fastai.callbacks.hooks import *

from fastai.callbacks.tracker import EarlyStoppingCallback
from fastai.callbacks.tracker import SaveModelCallback

path_wiki = Path('../input/wiki-face-data/wiki_crop/wiki_crop/')
path_imdb = Path('../input/imdb-wiki-faces-dataset/imdb_crop/imdb_crop/')
Collecting pretrainedmodels
  Downloading https://files.pythonhosted.org/packages/84/0e/be6a0e58447ac16c938799d49bfb5fb7a80ac35e137547fc6cee2c08c4cf/pretrainedmodels-0.7.4.tar.gz (58kB)
     |████████████████████████████████| 61kB 4.9MB/s eta 0:00:011
Requirement already satisfied: torch in /opt/conda/lib/python3.6/site-packages (from pretrainedmodels) (1.2.0)
Requirement already satisfied: torchvision in /opt/conda/lib/python3.6/site-packages (from pretrainedmodels) (0.4.0a0+6b959ee)
Requirement already satisfied: munch in /opt/conda/lib/python3.6/site-packages (from pretrainedmodels) (2.3.2)
Requirement already satisfied: tqdm in /opt/conda/lib/python3.6/site-packages (from pretrainedmodels) (4.36.1)
Requirement already satisfied: numpy in /opt/conda/lib/python3.6/site-packages (from torch->pretrainedmodels) (1.16.4)
Requirement already satisfied: six in /opt/conda/lib/python3.6/site-packages (from torchvision->pretrainedmodels) (1.12.0)
Requirement already satisfied: pillow>=4.1.1 in /opt/conda/lib/python3.6/site-packages (from torchvision->pretrainedmodels) (5.4.1)
Building wheels for collected packages: pretrainedmodels
  Building wheel for pretrainedmodels (setup.py) ... done
  Created wheel for pretrainedmodels: filename=pretrainedmodels-0.7.4-cp36-none-any.whl size=60963 sha256=b82608a0218b0ec36665a9794d3c75dedcac7ece90c42fab8c881ef88f2d2389
  Stored in directory: /tmp/.cache/pip/wheels/69/df/63/62583c096289713f22db605aa2334de5b591d59861a02c2ecd
Successfully built pretrainedmodels
Installing collected packages: pretrainedmodels
Successfully installed pretrainedmodels-0.7.4
Requirement already satisfied: fastai==1.0.57 in /opt/conda/lib/python3.6/site-packages (1.0.57)
Requirement already satisfied: packaging in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (19.2)
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (1.2.1)
Requirement already satisfied: beautifulsoup4 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (4.8.0)
Requirement already satisfied: pyyaml in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (5.1.2)
Requirement already satisfied: spacy>=2.0.18 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (2.1.8)
Requirement already satisfied: numexpr in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (2.6.9)
Requirement already satisfied: bottleneck in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (1.2.1)
Requirement already satisfied: matplotlib in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (3.0.3)
Requirement already satisfied: typing; python_version < "3.7" in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (3.6.6)
Requirement already satisfied: nvidia-ml-py3 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (7.352.0)
Requirement already satisfied: Pillow in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (5.4.1)
Requirement already satisfied: pandas in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (0.25.1)
Requirement already satisfied: torchvision in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (0.4.0a0+6b959ee)
Requirement already satisfied: fastprogress>=0.1.19 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (0.1.21)
Requirement already satisfied: dataclasses; python_version < "3.7" in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (0.6)
Requirement already satisfied: torch>=1.0.0 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (1.2.0)
Requirement already satisfied: numpy>=1.15 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (1.16.4)
Requirement already satisfied: requests in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (2.22.0)
Requirement already satisfied: six in /opt/conda/lib/python3.6/site-packages (from packaging->fastai==1.0.57) (1.12.0)
Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.6/site-packages (from packaging->fastai==1.0.57) (2.4.2)
Requirement already satisfied: soupsieve>=1.2 in /opt/conda/lib/python3.6/site-packages (from beautifulsoup4->fastai==1.0.57) (1.9.3)
Requirement already satisfied: blis<0.3.0,>=0.2.2 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (0.2.4)
Requirement already satisfied: thinc<7.1.0,>=7.0.8 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (7.0.8)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (0.9.6)
Requirement already satisfied: srsly<1.1.0,>=0.0.6 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (0.1.0)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (1.0.2)
Requirement already satisfied: wasabi<1.1.0,>=0.2.0 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (0.2.2)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (2.0.2)
Requirement already satisfied: preshed<2.1.0,>=2.0.1 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (2.0.1)
Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.6/site-packages (from matplotlib->fastai==1.0.57) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib->fastai==1.0.57) (1.1.0)
Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib->fastai==1.0.57) (2.8.0)
Requirement already satisfied: pytz>=2017.2 in /opt/conda/lib/python3.6/site-packages (from pandas->fastai==1.0.57) (2019.2)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.6/site-packages (from requests->fastai==1.0.57) (2019.9.11)
Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.6/site-packages (from requests->fastai==1.0.57) (2.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.6/site-packages (from requests->fastai==1.0.57) (1.24.2)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.6/site-packages (from requests->fastai==1.0.57) (3.0.4)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /opt/conda/lib/python3.6/site-packages (from thinc<7.1.0,>=7.0.8->spacy>=2.0.18->fastai==1.0.57) (4.36.1)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib->fastai==1.0.57) (41.2.0)
import scipy.io
import numpy as np
import pandas as pd
from datetime import datetime, timedelta

Loading the Data

mat_wiki = scipy.io.loadmat('../input/wiki-face-data/wiki_crop/wiki_crop/wiki.mat')
mat_imdb = scipy.io.loadmat('../input/imdb-wiki-faces-dataset/imdb_crop/imdb_crop/imdb.mat')
columns = ["dob", "photo_taken", "full_path", "gender", "name", "face_location", 
           "face_score", "second_face_score", 'celeb_names', 'celeb_id']
instances_wiki = mat_wiki['wiki'][0][0][0].shape[1]
instances_imdb = mat_imdb['imdb'][0][0][0].shape[1]


df_wiki = pd.DataFrame(index = range(0,instances_wiki), columns = columns)
df_imdb = pd.DataFrame(index = range(0,instances_imdb), columns = columns)
for i in mat_wiki:
    if i == "wiki":
        current_array = mat_wiki[i][0][0]
        for j in range(len(current_array)):
            #print(columns[j],": ",current_array[j])
            df_wiki[columns[j]] = pd.DataFrame(current_array[j][0])
            

for i in mat_imdb:
    if i == "imdb":
        current_array = mat_imdb[i][0][0]
        for j in range(len(current_array)):
            #print(columns[j],": ",current_array[j])
            df_imdb[columns[j]] = pd.DataFrame(current_array[j][0])
df_wiki.head()
dob photo_taken full_path gender name face_location face_score second_face_score celeb_names celeb_id
0 723671 2009 [17/10000217_1981-05-05_2009.jpg] 1.0 [Sami Jauhojärvi] [[111.29109473290997, 111.29109473290997, 252.... 4.300962 NaN NaN NaN
1 703186 1964 [48/10000548_1925-04-04_1964.jpg] 1.0 [Dettmar Cramer] [[252.48330229530742, 126.68165114765371, 354.... 2.645639 1.949248 NaN NaN
2 711677 2008 [12/100012_1948-07-03_2008.jpg] 1.0 [Marc Okrand] [[113.52, 169.83999999999997, 366.08, 422.4]] 4.329329 NaN NaN NaN
3 705061 1961 [65/10001965_1930-05-23_1961.jpg] 1.0 [Aleksandar Matanović] [[1, 1, 634, 440]] -inf NaN NaN NaN
4 720044 2012 [16/10002116_1971-05-31_2012.jpg] 0.0 [Diana Damrau] [[171.61031405173117, 75.57451239763239, 266.7... 3.408442 NaN NaN NaN
df_imdb.head()
dob photo_taken full_path gender name face_location face_score second_face_score celeb_names celeb_id
0 693726 1968 [01/nm0000001_rm124825600_1899-5-10_1968.jpg] 1.0 [Fred Astaire] [[1072.926, 161.838, 1214.7839999999999, 303.6... 1.459693 1.118973 ['Lee' George Quinones] 6488
1 693726 1970 [01/nm0000001_rm3343756032_1899-5-10_1970.jpg] 1.0 [Fred Astaire] [[477.184, 100.352, 622.592, 245.76]] 2.543198 1.852008 ['Weird Al' Yankovic] 6488
2 693726 1968 [01/nm0000001_rm577153792_1899-5-10_1968.jpg] 1.0 [Fred Astaire] [[114.96964308962852, 114.96964308962852, 451.... 3.455579 2.985660 [2 Chainz] 6488
3 693726 1968 [01/nm0000001_rm946909184_1899-5-10_1968.jpg] 1.0 [Fred Astaire] [[622.8855056426588, 424.21750383700805, 844.3... 1.872117 NaN [50 Cent] 6488
4 693726 1968 [01/nm0000001_rm980463616_1899-5-10_1968.jpg] 1.0 [Fred Astaire] [[1013.8590023603723, 233.8820422075853, 1201.... 1.158766 NaN [A Martinez] 6488
df_wiki.shape, df_imdb.shape
((62328, 10), (460723, 10))
def datenum_to_datetime(datenum):
    
    try:
        days = datenum % 1
        hours = days % 1 * 24
        minutes = hours % 1 * 60
        seconds = minutes % 1 * 60
        exact_date = datetime.fromordinal(int(datenum)) \
           + timedelta(days=int(days)) \
           + timedelta(hours=int(hours)) \
           + timedelta(minutes=int(minutes)) \
           + timedelta(seconds=round(seconds)) \
           - timedelta(days=366)
    
        return exact_date.year
    
    except(ValueError, TypeError, OverflowError):
        
        return np.nan  
df_wiki['date_of_birth'] = df_wiki['dob'].apply(datenum_to_datetime) 
df_imdb['date_of_birth'] = df_imdb['dob'].apply(datenum_to_datetime) 
df_wiki['date_of_birth'].value_counts()
df_imdb['date_of_birth'].value_counts()
1969.0    16511
1978.0    16062
1971.0    15344
1976.0    14232
1970.0    14184
          ...  
1879.0        1
2012.0        1
1883.0        1
1888.0        1
1732.0        1
Name: date_of_birth, Length: 128, dtype: int64
df_wiki['age'] = df_wiki['photo_taken'] - df_wiki['date_of_birth']

#remove pictures does not include face
df_wiki = df_wiki[df_wiki['face_score'] != -np.inf]

#some pictures include more than one face, remove them
df_wiki = df_wiki[df_wiki['second_face_score'].isna()]

#check threshold
df_wiki = df_wiki[df_wiki['face_score'] >= 3.5]

df_wiki = df_wiki.drop(columns = ['name','face_score','second_face_score','date_of_birth','face_location'])

#some guys seem to be greater than 100. some of these are paintings. remove these old guys
df_wiki = df_wiki[df_wiki['age'] <= 100]

#some guys seem to be unborn in the data set
df_wiki = df_wiki[df_wiki['age'] > 0]
df_imdb['age'] = df_imdb['photo_taken'] - df_imdb['date_of_birth']

#remove pictures does not include face
df_imdb = df_imdb[df_imdb['face_score'] != -np.inf]

#some pictures include more than one face, remove them
df_imdb = df_imdb[df_imdb['second_face_score'].isna()]

#check threshold
df_imdb = df_imdb[df_imdb['face_score'] >= 3.5]

df_imdb = df_imdb.drop(columns = ['name','face_score','second_face_score','date_of_birth','face_location'])

#some guys seem to be greater than 100. some of these are paintings. remove these old guys
df_imdb = df_imdb[df_imdb['age'] <= 100]

#some guys seem to be unborn in the data set
df_imdb = df_imdb[df_imdb['age'] > 0]
df_wiki.head()
dob photo_taken full_path gender celeb_names celeb_id age
0 723671 2009 [17/10000217_1981-05-05_2009.jpg] 1.0 NaN NaN 28
2 711677 2008 [12/100012_1948-07-03_2008.jpg] 1.0 NaN NaN 60
5 716189 2012 [02/10002702_1960-11-09_2012.jpg] 0.0 NaN NaN 52
6 707745 1971 [41/10003541_1937-09-27_1971.jpg] 1.0 NaN NaN 34
10 697114 1950 [99/10004299_1908-08-19_1950.jpg] 1.0 NaN NaN 42
df_wiki.shape, df_imdb.shape
((17096, 7), (73683, 7))
df_wiki['age'] = df_wiki['age'].apply(lambda x: int(x))
df_imdb['age'] = df_imdb['age'].apply(lambda x: int(x))
print(type(df_wiki['age']))
df_wiki['age'].value_counts()
df_imdb['age'].value_counts()
<class 'pandas.core.series.Series'>
31    2757
32    2677
29    2578
33    2545
30    2532
      ... 
96       1
95       1
93       1
92       1
3        1
Name: age, Length: 98, dtype: int64
df_wiki = df_wiki.drop(columns=['dob', 'photo_taken'])
df_imdb = df_imdb.drop(columns=['dob', 'photo_taken'])
df_age_wiki = df_wiki.drop(columns=['gender', 'celeb_names', 'celeb_id'])
df_age_imdb = df_imdb.drop(columns=['gender', 'celeb_names', 'celeb_id'])
df_age_wiki['age'].value_counts()
24    685
25    658
26    658
27    651
23    649
     ... 
97      1
6       1
5       1
94      1
95      1
Name: age, Length: 97, dtype: int64
df_imdb['age'].value_counts()
31    2757
32    2677
29    2578
33    2545
30    2532
      ... 
96       1
95       1
93       1
92       1
3        1
Name: age, Length: 98, dtype: int64
df_age_wiki['full_path'] = df_age_wiki['full_path'].str.get(0)
df_age_imdb['full_path'] = df_age_imdb['full_path'].str.get(0)
df_age_wiki.dropna(axis=0, inplace=True)
df_age_imdb.dropna(axis=0, inplace=True)
df_age_wiki.head()
full_path age
0 17/10000217_1981-05-05_2009.jpg 28
2 12/100012_1948-07-03_2008.jpg 60
5 02/10002702_1960-11-09_2012.jpg 52
6 41/10003541_1937-09-27_1971.jpg 34
10 99/10004299_1908-08-19_1950.jpg 42
df_age_imdb.head()
full_path age
7 02/nm0000002_rm1363385088_1924-9-16_2004.jpg 80
12 02/nm0000002_rm221957120_1924-9-16_1974.jpg 50
14 02/nm0000002_rm238734336_1924-9-16_1974.jpg 50
15 02/nm0000002_rm2585828096_1924-9-16_2006.jpg 82
18 02/nm0000002_rm2769394176_1924-9-16_2006.jpg 82
df_age_wiki['age'] = df_age_wiki['age'].apply(lambda x: int(x))
df_age_imdb['age'] = df_age_imdb['age'].apply(lambda x: int(x))
df_age_wiki.shape, df_age_imdb.shape
((17096, 2), (73683, 2))
max_age = df_age_wiki["age"].max(); print(max_age)
min_age = df_age_wiki["age"].min(); print(min_age)
100
1
max_age = df_age_imdb["age"].max(); print(max_age)
min_age = df_age_imdb["age"].min(); print(min_age)
99
1
df_age_wiki.hist()
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7fcf17fc1278>]], dtype=object)
df_age_imdb.hist()
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7fcf163a30f0>]], dtype=object)
seed = 42

# # python RNG
# import random
# random.seed(seed)

# # pytorch RNGs
# import torch
# torch.manual_seed(seed)
# torch.backends.cudnn.deterministic = True
# if torch.cuda.is_available(): torch.cuda.manual_seed_all(seed)

# numpy RNG
import numpy as np
np.random.seed(seed)
path = Path('../input/')
path_utk = Path('../input/utk-face-cropped/utkcropped/utkcropped/')
def extract_age(filename):
    return float(filename.stem.split('_')[0])

Preparing the Data for Fastai Models

tfms = get_transforms(max_rotate= 10.,max_zoom=1., max_lighting=0.20, do_flip=False,
                      max_warp=0., xtra_tfms=[flip_lr(), brightness(change=(0.3, 0.60), p=0.7), contrast(scale=(0.5, 2), p=0.7),
                                              crop_pad(size=600, padding_mode='border', row_pct=0.,col_pct=0.),
                                              rand_zoom(scale=(1.,1.5)), rand_crop(),
                                              perspective_warp(magnitude=(-0.1,0.1)),
                                              #jitter(magnitude=(-0.05,0.05), p=0.5),
                                              symmetric_warp(magnitude=(-0.1,0.1)) ])

path_imdb = Path('../input/imdb-wiki-faces-dataset/imdb_crop/imdb_crop/')
path_wiki = Path('../input/wiki-face-data/wiki_crop/wiki_crop/')


data_imdb = ImageList.from_df(df_age_imdb, path_imdb, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_wiki_small = ImageList.from_df(df_age_wiki, path_wiki, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_wiki_big = ImageList.from_df(df_age_wiki, path_wiki, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_utk_small = ImageList.from_folder(path_utk).split_by_rand_pct(0.2, seed=42).label_from_func(extract_age, label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_utk_big = ImageList.from_folder(path_utk).split_by_rand_pct(0.2, seed=42).label_from_func(extract_age, label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
df_appa = pd.read_csv('/kaggle/input/appa-real-face-cropped/labels.csv')
df_appa.head()
file_name real_age
0 000000.jpg 4
1 000001.jpg 18
2 000002.jpg 80
3 000003.jpg 50
4 000004.jpg 17
df_appa.rename(columns = {"file_name":"full_path", "real_age":"age"}, inplace=True)
df_appa['age'] = df_appa['age'].apply(lambda x: int(float(x)))
path = Path('../input/appa-real-face-cropped/final_files/final_files/')
path_csv = '../input/appa-real-face-cropped/'
path_folder = '../input/appa-real-face-cropped/final_files/final_files/'

data_appa_small = ImageList.from_df(df_appa, path, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_appa_big = ImageList.from_df(df_appa, path, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)

A look at the Images

data_imdb.show_batch(4, figsize=(12,12))
data_wiki_small.show_batch(4, figsize=(12,12))
data_wiki_big.show_batch(4, figsize=(12,12))
data_utk_small.show_batch(4, figsize=(12,12))
data_utk_big.show_batch(4, figsize=(12,12))
data_appa_small.show_batch(4, figsize=(12,12))
data_appa_big.show_batch(4, figsize=(12,12))

Fastai Modelling

import torch
import torch.nn as nn
import torch.nn.functional as F
from pathlib import Path
import random
opt_func = partial(optim.Adam, betas=(0.9,0.99), eps=1e-5)
def resnet(pretrained=False):
    pretrained = 'imagenet' if pretrained else None
    model = pretrainedmodels.resnet34(pretrained=pretrained)
    return model
class L1LossFlat(nn.SmoothL1Loss):
    def forward(self, input:Tensor, target:Tensor) -> Rank0Tensor:
        return super().forward(input.view(-1), target.view(-1))
df_utk_small = data_utk_small.to_df()
df_utk_small.head()
x y
0 39_0_1_20170116143117964.jpg.chip.jpg 39.0
1 54_0_3_20170104202315035.jpg.chip.jpg 54.0
2 1_0_2_20161219154715885.jpg.chip.jpg 1.0
3 30_1_2_20170116192313635.jpg.chip.jpg 30.0
4 41_1_0_20170117133203334.jpg.chip.jpg 41.0
df_utk_small.rename(columns = {"x":"full_path", "y":"age"}, inplace=True)
df_utk_small['age'] = df_utk_small['age'].apply(lambda x: int(float(x)))
df_utk_small.head()
full_path age
0 39_0_1_20170116143117964.jpg.chip.jpg 39
1 54_0_3_20170104202315035.jpg.chip.jpg 54
2 1_0_2_20161219154715885.jpg.chip.jpg 1
3 30_1_2_20170116192313635.jpg.chip.jpg 30
4 41_1_0_20170117133203334.jpg.chip.jpg 41
df_appa.head()
full_path age
0 000000.jpg 4
1 000001.jpg 18
2 000002.jpg 80
3 000003.jpg 50
4 000004.jpg 17
src_wiki = '../input/wiki-face-data/wiki_crop/wiki_crop/'
src_utk = '../input/utk-face-cropped/utkcropped/utkcropped/'
src_appa = '../input/appa-real-face-cropped/final_files/'

dest = '../working/wiki_utk/'

import os
import shutil
import pathlib

pathlib.Path(dest).mkdir(parents=True, exist_ok=True)

os.listdir(dest)
[]
for root, dirs, files in os.walk(src_wiki):
    for file in files:
        path_file = os.path.join(root,file)
        shutil.copy2(path_file, dest)
for root, dirs, files in os.walk(src_utk):
    for file in files:
        path_file = os.path.join(root,file)
        shutil.copy2(path_file,dest)
for root, dirs, files in os.walk(src_appa):
    for file in files:
        path_file = os.path.join(root,file)
        shutil.copy2(path_file,dest)
df_age_wiki.head()
full_path age
0 17/10000217_1981-05-05_2009.jpg 28
2 12/100012_1948-07-03_2008.jpg 60
5 02/10002702_1960-11-09_2012.jpg 52
6 41/10003541_1937-09-27_1971.jpg 34
10 99/10004299_1908-08-19_1950.jpg 42
df_age_wiki['full_path'] = df_age_wiki['full_path'].str[3:]
df_age_wiki.head()
full_path age
0 10000217_1981-05-05_2009.jpg 28
2 100012_1948-07-03_2008.jpg 60
5 10002702_1960-11-09_2012.jpg 52
6 10003541_1937-09-27_1971.jpg 34
10 10004299_1908-08-19_1950.jpg 42
frames = [df_age_wiki, df_utk_small, df_appa]
df_wiki_utk_appa = pd.concat(frames)
df_wiki_utk_appa.head()
full_path age
0 10000217_1981-05-05_2009.jpg 28
2 100012_1948-07-03_2008.jpg 60
5 10002702_1960-11-09_2012.jpg 52
6 10003541_1937-09-27_1971.jpg 34
10 10004299_1908-08-19_1950.jpg 42
df_wiki_utk_appa.shape
(43655, 2)
os.listdir(dest)
['29_1_1_20170112234606137.jpg.chip.jpg',
 '48_1_0_20170105000634011.jpg.chip.jpg',
 '32_0_1_20170113001322715.jpg.chip.jpg',
 '26_0_0_20170116192554607.jpg.chip.jpg',
 '36561562_1888-07-24_2004.jpg',
 '2338017_1960-09-21_2002.jpg',
 '3484443_1981-06-17_2006.jpg',
 '35_1_1_20170113003536772.jpg.chip.jpg',
 '52517_1934-10-07_1964.jpg',
 '46348054_1920-08-10_1942.jpg',
 '15_1_2_20170102234824195.jpg.chip.jpg',
 '903342_1972-06-28_2008.jpg',
 '29_0_1_20170117094745480.jpg.chip.jpg',
 '3762844_1945-09-25_1978.jpg',
 '27846664_1901-11-01_2010.jpg',
 '1051405_1969-11-19_2007.jpg',
 '249274_1955-07-01_2012.jpg',
 '20053572_1982-04-06_2009.jpg',
 '24815184_1864-08-15_2004.jpg',
 '001081.jpg',
 '12653807_1970-07-21_2011.jpg',
 '1_1_0_20170109193704430.jpg.chip.jpg',
 '26537728_1951-10-11_1978.jpg',
 '9952833_1990-06-05_2013.jpg',
 '88_0_1_20170116144533236.jpg.chip.jpg',
 '9632534_1957-06-09_2000.jpg',
 '36_1_0_20170104172916913.jpg.chip.jpg',
 '7796_1884-06-21_1941.jpg',
 '39160244_1989-03-05_2012.jpg',
 '50_1_0_20170117092638126.jpg.chip.jpg',
 '29_1_0_20170104172655682.jpg.chip.jpg',
 '7118448_1972-09-17_2008.jpg',
 '38372732_1939-06-09_1960.jpg',
 '35990083_1960-11-24_2001.jpg',
 '40_1_1_20170112231504476.jpg.chip.jpg',
 '27_1_2_20170116170956198.jpg.chip.jpg',
 '38_0_0_20170117175858084.jpg.chip.jpg',
 '28449525_1950-03-17_1994.jpg',
 '682290_1951-07-24_2009.jpg',
 '30079345_1993-07-20_2014.jpg',
 '46776424_1890-12-19_1944.jpg',
 '24_1_1_20170112204751139.jpg.chip.jpg',
 '26_0_3_20170119181310597.jpg.chip.jpg',
 '198400_1913-02-19_1961.jpg',
 '29200196_1970-01-01_2014.jpg',
 '25842166_1987-05-23_2011.jpg',
 '24576620_1956-08-20_2010.jpg',
 '000244.jpg',
 '46243758_1925-03-26_1990.jpg',
 '18767507_1931-05-01_2008.jpg',
 '99_1_0_20170120134718759.jpg.chip.jpg',
 '12970907_1930-10-16_1948.jpg',
 '001372.jpg',
 '378848_1945-08-03_2013.jpg',
 '20751442_1988-03-02_2012.jpg',
 '2446664_1978-10-26_2008.jpg',
 '8869997_1981-02-13_2012.jpg',
 '32504267_1954-04-17_1979.jpg',
 '19186371_1985-06-10_2009.jpg',
 '34430247_1896-12-30_1944.jpg',
 '27_1_1_20170113001031147.jpg.chip.jpg',
 '1_0_3_20161220143121101.jpg.chip.jpg',
 '501287_1955-05-10_2007.jpg',
 '65_0_0_20170111193809606.jpg.chip.jpg',
 '46596624_1960-02-16_1979.jpg',
 '14950334_1971-05-26_2006.jpg',
 '18418711_1904-10-16_1952.jpg',
 '5853885_1969-08-08_2014.jpg',
 '36836003_1964-12-19_1990.jpg',
 '853348_1931-10-26_2012.jpg',
 '40_0_0_20170117181229922.jpg.chip.jpg',
 '5_1_4_20161221201952681.jpg.chip.jpg',
 '4877949_1983-10-27_2013.jpg',
 '1662630_1971-05-01_2005.jpg',
 '4498970_1994-02-27_2013.jpg',
 '24_1_2_20170116171627787.jpg.chip.jpg',
 '15187291_1901-05-31_1943.jpg',
 '1096879_1945-09-06_2005.jpg',
 '39_0_3_20170104220721736.jpg.chip.jpg',
 '28707402_1988-12-30_2012.jpg',
 '1_0_0_20170109191453449.jpg.chip.jpg',
 '42361164_1995-12-14_2014.jpg',
 '34564786_1961-09-13_2007.jpg',
 '21_1_1_20170114032154585.jpg.chip.jpg',
 '28120539_1989-06-25_2009.jpg',
 '1274866_1939-01-06_1985.jpg',
 '25_1_0_20170117195346437.jpg.chip.jpg',
 '24353667_1924-04-01_1950.jpg',
 '003526.jpg',
 '24086813_1923-05-13_1943.jpg',
 '2066719_1983-06-20_2013.jpg',
 '32_0_1_20170116215755940.jpg.chip.jpg',
 '006293.jpg',
 '006260.jpg',
 '603566_1940-07-18_2012.jpg',
 '950281_1967-10-01_2004.jpg',
 '30_1_1_20170114024643760.jpg.chip.jpg',
 '003307.jpg',
 '000010.jpg',
 '58_0_0_20170113185223064.jpg.chip.jpg',
 '5340721_1945-08-19_2013.jpg',
 '4462806_1980-11-13_2011.jpg',
 '21750900_1984-06-06_2009.jpg',
 '1_0_2_20161219221816327.jpg.chip.jpg',
 '15685887_1942-09-11_1966.jpg',
 '9236375_1971-09-02_2010.jpg',
 '17_1_3_20170104221735351.jpg.chip.jpg',
 '15823114_1954-10-06_2006.jpg',
 '1573777_1943-12-31_2005.jpg',
 '3007277_1954-09-07_2013.jpg',
 '38479138_1986-04-23_2006.jpg',
 '006755.jpg',
 '1480772_1942-09-26_1990.jpg',
 '42244963_1948-04-19_2014.jpg',
 '51_0_0_20170120222842138.jpg.chip.jpg',
 '36422331_1982-02-28_2012.jpg',
 '7895818_1927-04-06_1960.jpg',
 '44744914_1941-02-22_2007.jpg',
 '796076_1969-05-16_2010.jpg',
 '13106750_1986-09-27_2011.jpg',
 '24_1_2_20170116165633882.jpg.chip.jpg',
 '61_1_2_20170105174916768.jpg.chip.jpg',
 '1567044_1969-10-24_2009.jpg',
 '3614080_1965-08-10_2013.jpg',
 '11775419_1978-07-26_2008.jpg',
 '58_0_3_20170119211559343.jpg.chip.jpg',
 '21178468_1951-12-20_1973.jpg',
 '004508.jpg',
 '006975.jpg',
 '40800834_1971-01-31_2010.jpg',
 '55_0_0_20170117185729852.jpg.chip.jpg',
 '37201426_1985-02-03_2014.jpg',
 '26788886_1962-10-14_1984.jpg',
 '37_0_0_20170109013120280.jpg.chip.jpg',
 '1_1_2_20161219201453804.jpg.chip.jpg',
 '39611913_1993-02-15_2014.jpg',
 '38379760_1990-05-14_2013.jpg',
 '29351931_1927-01-20_1990.jpg',
 '30_1_0_20170116201303368.jpg.chip.jpg',
 '10016074_1987-07-27_2012.jpg',
 '24279039_1981-04-23_2008.jpg',
 '30687645_1992-05-29_2011.jpg',
 '44285586_1992-06-19_2012.jpg',
 '51_0_0_20170111171747340.jpg.chip.jpg',
 '16534354_1989-08-21_2012.jpg',
 '28_0_3_20170119194803955.jpg.chip.jpg',
 '37128181_1992-01-13_2012.jpg',
 '34686067_1921-09-09_1960.jpg',
 '45422904_1996-03-05_2015.jpg',
 '37126092_1991-07-06_2014.jpg',
 '42520670_1940-02-11_1960.jpg',
 '89_0_1_20170117193440263.jpg.chip.jpg',
 '37_0_4_20170104000748917.jpg.chip.jpg',
 '43619465_1989-03-22_2014.jpg',
 '004828.jpg',
 '004922.jpg',
 '1624907_1927-08-07_2007.jpg',
 '20016339_1992-02-09_2009.jpg',
 '29_0_1_20170108235740510.jpg.chip.jpg',
 '25_1_1_20170112205103738.jpg.chip.jpg',
 '31_1_3_20170109134554005.jpg.chip.jpg',
 '779097_1977-03-06_2008.jpg',
 '25_0_2_20170117195218570.jpg.chip.jpg',
 '8027145_1978-07-25_1998.jpg',
 '2443611_1976-05-25_2007.jpg',
 '87_0_1_20170117194805270.jpg.chip.jpg',
 '2026045_1955-05-23_2007.jpg',
 '15803680_1942-12-26_2008.jpg',
 '1712978_1977-12-10_2008.jpg',
 '21644506_1985-07-05_2011.jpg',
 '6733489_1944-11-02_1968.jpg',
 '32432553_1922-02-26_1947.jpg',
 '000381.jpg',
 '3354295_1995-07-09_2008.jpg',
 '40890850_1949-07-20_2013.jpg',
 '2872703_1965-09-24_2012.jpg',
 '19447796_1989-07-26_2012.jpg',
 '10977057_1980-11-24_2008.jpg',
 '6145763_1960-02-19_2008.jpg',
 '35584490_1990-01-31_2012.jpg',
 '204942_1955-07-14_2011.jpg',
 '33947707_1995-12-29_2013.jpg',
 '28285893_1992-02-25_2011.jpg',
 '4_0_0_20170110213530901.jpg.chip.jpg',
 '2507966_1973-08-03_2014.jpg',
 '005261.jpg',
 '32_1_0_20170117174838232.jpg.chip.jpg',
 '1706281_1942-06-16_2003.jpg',
 '16617089_1963-03-09_2009.jpg',
 '39211197_1943-02-24_2006.jpg',
 '30501540_1960-07-29_1990.jpg',
 '25854312_1968-05-02_2010.jpg',
 '45_0_3_20170119203905073.jpg.chip.jpg',
 '14808073_1952-08-25_2015.jpg',
 '002745.jpg',
 '1633417_1912-02-20_1982.jpg',
 '31789140_1980-02-25_2000.jpg',
 '35_1_1_20170113005402981.jpg.chip.jpg',
 '2802601_1927-03-13_2002.jpg',
 '29462622_1935-06-10_1960.jpg',
 '18115101_1946-06-18_1941.jpg',
 '42_0_1_20170117171503046.jpg.chip.jpg',
 '30878743_1977-11-29_2000.jpg',
 '1234991_1912-03-22_1952.jpg',
 '006942.jpg',
 '19487892_1983-10-11_2015.jpg',
 '9427912_1969-10-23_2006.jpg',
 '36_0_0_20170109003634534.jpg.chip.jpg',
 '23617680_1929-09-28_1971.jpg',
 '7976847_1985-07-30_2011.jpg',
 '42822079_1970-11-30_2013.jpg',
 '79_0_0_20170111205953011.jpg.chip.jpg',
 '37732462_1993-07-28_2012.jpg',
 '58_0_0_20170117191839091.jpg.chip.jpg',
 '30_0_1_20170113155013364.jpg.chip.jpg',
 '1262985_1932-05-19_1963.jpg',
 '750603_1942-10-21_2012.jpg',
 '18600259_1982-02-28_2010.jpg',
 '29582036_1988-06-21_2008.jpg',
 '2_1_3_20161220144844310.jpg.chip.jpg',
 '28_1_0_20170117092254042.jpg.chip.jpg',
 '36967227_1897-12-02_2008.jpg',
 '1_0_3_20161219225759904.jpg.chip.jpg',
 '52_0_0_20170104204156747.jpg.chip.jpg',
 '46498373_1938-01-26_1959.jpg',
 '5197788_1959-12-31_2005.jpg',
 '14307202_1985-01-16_2007.jpg',
 '13498620_1985-06-25_2014.jpg',
 '1467908_1927-01-01_1948.jpg',
 '6777493_1985-01-09_2013.jpg',
 '38842697_1993-08-09_2014.jpg',
 '17264219_1933-04-03_1956.jpg',
 '7725612_1974-07-07_2011.jpg',
 '002618.jpg',
 '4_0_0_20170110213253207.jpg.chip.jpg',
 '30268663_1970-03-01_2013.jpg',
 '29797248_1989-10-22_2013.jpg',
 '32_1_3_20170119155703795.jpg.chip.jpg',
 '40519566_1941-01-10_1975.jpg',
 '004994.jpg',
 '28790298_1988-03-20_2011.jpg',
 '32888889_1976-12-15_2014.jpg',
 '13069966_1959-05-09_2006.jpg',
 '003129.jpg',
 '201160_1914-09-15_1968.jpg',
 '8347154_1935-12-09_1955.jpg',
 '28732281_1986-09-24_2010.jpg',
 '21346551_1931-01-09_1958.jpg',
 '2486630_1948-01-27_2006.jpg',
 '476099_1946-02-05_2009.jpg',
 '8006688_1961-03-14_2007.jpg',
 '4442797_1971-09-16_2014.jpg',
 '29950756_1986-02-28_2015.jpg',
 '16_0_0_20170110231218369.jpg.chip.jpg',
 '11496750_1930-05-19_1986.jpg',
 '44681795_1984-09-13_2015.jpg',
 '3591776_1946-12-31_1972.jpg',
 '3_1_4_20161221193144389.jpg.chip.jpg',
 '1984786_1930-11-23_2004.jpg',
 '4065567_1942-08-26_1998.jpg',
 '39535197_1988-11-28_2014.jpg',
 '1982061_1975-03-15_2012.jpg',
 '2593429_1947-09-19_1976.jpg',
 '769156_1882-02-04_1944.jpg',
 '45555747_1956-06-03_2015.jpg',
 '88833_1916-07-01_1945.jpg',
 '38032109_1992-04-18_2015.jpg',
 '002553.jpg',
 '7052971_1978-08-15_2010.jpg',
 '30760593_1946-02-05_1968.jpg',
 '26_0_1_20170113135516970.jpg.chip.jpg',
 '870163_1899-09-22_1948.jpg',
 '004412.jpg',
 '6056366_1954-11-11_1950.jpg',
 '28624328_1953-08-23_1973.jpg',
 '55_0_0_20170120221856526.jpg.chip.jpg',
 '1607852_1976-08-06_2007.jpg',
 '21_1_4_20170103223204087.jpg.chip.jpg',
 '10982693_1955-12-10_1992.jpg',
 '17958040_1987-07-24_2011.jpg',
 '20670994_1961-05-03_1988.jpg',
 '32216891_1935-05-24_1960.jpg',
 '67291_1937-03-22_1960.jpg',
 '50_0_0_20170117153725428.jpg.chip.jpg',
 '44_1_1_20170117142913571.jpg.chip.jpg',
 '004467.jpg',
 '24_1_0_20170117150535820.jpg.chip.jpg',
 '1_0_2_20161219151026723.jpg.chip.jpg',
 '9_0_2_20170110224822473.jpg.chip.jpg',
 '40_0_0_20170116221022897.jpg.chip.jpg',
 '35798200_1987-08-03_2014.jpg',
 '002576.jpg',
 '222737_1955-10-12_2007.jpg',
 '25644669_1953-10-28_2009.jpg',
 '45_0_0_20170116212320846.jpg.chip.jpg',
 '5841825_1985-03-15_2012.jpg',
 '60_0_0_20170120223043268.jpg.chip.jpg',
 '46340639_1945-04-03_1969.jpg',
 '17256613_1950-04-21_1984.jpg',
 '20050887_1973-07-21_1996.jpg',
 '005593.jpg',
 '46730387_1922-05-27_2013.jpg',
 '33415719_1937-03-23_1962.jpg',
 '24_1_0_20170117150310308.jpg.chip.jpg',
 '719076_1960-09-04_2012.jpg',
 '14146965_1983-11-27_2013.jpg',
 '7391254_1936-08-21_1962.jpg',
 '38594467_1992-03-08_2014.jpg',
 '32222137_1984-11-24_2010.jpg',
 '27_1_1_20170116214220570.jpg.chip.jpg',
 '10046706_1933-04-21_1962.jpg',
 '000005.jpg',
 '32_1_0_20170103180822745.jpg.chip.jpg',
 '30049841_1969-11-04_2015.jpg',
 '429097_1954-06-02_2007.jpg',
 '85_0_0_20170111210559755.jpg.chip.jpg',
 '001082.jpg',
 '3062692_1965-07-31_2011.jpg',
 '52_0_0_20170117135533022.jpg.chip.jpg',
 '32227332_1992-08-24_2015.jpg',
 '1_0_2_20161219204958037.jpg.chip.jpg',
 '1809170_1967-08-01_2011.jpg',
 '20816070_1971-05-10_2007.jpg',
 '35_1_0_20170103182732026.jpg.chip.jpg',
 '27637550_1917-03-10_2007.jpg',
 '15_0_0_20170104013333801.jpg.chip.jpg',
 '39961436_1925-01-26_1948.jpg',
 '21291667_1913-10-02_1950.jpg',
 '006552.jpg',
 '40492912_1988-06-30_2013.jpg',
 '42451669_1983-04-12_2012.jpg',
 '329524_1945-10-24_2014.jpg',
 '24_1_1_20170112192919494.jpg.chip.jpg',
 '8_0_0_20170110220118138.jpg.chip.jpg',
 '38416948_1962-07-13_1991.jpg',
 '33791098_1979-10-04_2011.jpg',
 '28_0_1_20170113142308413.jpg.chip.jpg',
 '40210503_1992-08-30_2014.jpg',
 '9509458_1993-01-31_2008.jpg',
 '39098401_1921-10-29_1952.jpg',
 '23_1_0_20170117144740562.jpg.chip.jpg',
 '42_1_2_20170116162349143.jpg.chip.jpg',
 '41978653_1951-07-27_2008.jpg',
 '88371_1944-05-20_2013.jpg',
 '44_1_3_20170119172158321.jpg.chip.jpg',
 '20329296_1983-10-13_2008.jpg',
 '001972.jpg',
 '31976668_1991-03-02_2015.jpg',
 '007118.jpg',
 '32752294_1973-02-07_1997.jpg',
 '50_1_0_20170117155139407.jpg.chip.jpg',
 '002627.jpg',
 '39220677_1976-04-22_1998.jpg',
 '26_0_3_20170119180948868.jpg.chip.jpg',
 '5101720_1975-06-30_2014.jpg',
 '34686543_1948-01-09_1992.jpg',
 '14276461_1983-04-18_2011.jpg',
 '5445094_1936-08-13_2012.jpg',
 '7_0_2_20161219193457075.jpg.chip.jpg',
 '1_1_2_20161219162243182.jpg.chip.jpg',
 '3060227_1974-10-02_2012.jpg',
 '39903017_1989-08-26_2013.jpg',
 '27962750_1894-03-20_1946.jpg',
 '42149298_1978-09-03_2014.jpg',
 '006396.jpg',
 '27_0_0_20170116204136393.jpg.chip.jpg',
 '10007577_1988-06-16_2007.jpg',
 '44835846_1946-12-08_1978.jpg',
 '26_1_1_20170116162111804.jpg.chip.jpg',
 '13713930_1976-06-27_2012.jpg',
 '27_1_2_20170116163622032.jpg.chip.jpg',
 '004115.jpg',
 '27_1_3_20170109132029072.jpg.chip.jpg',
 '003751.jpg',
 '9201851_1923-10-10_1953.jpg',
 '27_1_4_20170103235051403.jpg.chip.jpg',
 '13551558_1980-09-17_2010.jpg',
 '15483779_1950-05-02_1950.jpg',
 '1057264_1947-03-06_1974.jpg',
 '001200.jpg',
 '32_1_1_20170112212952111.jpg.chip.jpg',
 '41942456_1987-06-18_2014.jpg',
 '40_1_0_20170117160718607.jpg.chip.jpg',
 '3538953_1967-05-14_2013.jpg',
 '5928640_1957-06-14_2013.jpg',
 '18933035_1982-08-26_2012.jpg',
 '27_1_0_20170117120642103.jpg.chip.jpg',
 '37_0_1_20170117134417970.jpg.chip.jpg',
 '42_1_0_20170117121915102.jpg.chip.jpg',
 '3603119_1940-08-28_2011.jpg',
 '4163901_1950-08-20_1974.jpg',
 '23814332_1990-02-21_2007.jpg',
 '35_1_0_20170117170314803.jpg.chip.jpg',
 '35_0_3_20170104214246948.jpg.chip.jpg',
 '2415032_1954-02-19_2012.jpg',
 '8277549_1979-11-06_2005.jpg',
 '44117991_1918-02-14_1944.jpg',
 '42_0_0_20170104184335702.jpg.chip.jpg',
 '004633.jpg',
 '27857682_1916-11-13_1970.jpg',
 '14475885_1953-02-25_2010.jpg',
 '17_0_0_20170114025913647.jpg.chip.jpg',
 '36_1_1_20170113011506296.jpg.chip.jpg',
 '297985_1911-07-16_1955.jpg',
 '44400523_1928-07-26_1971.jpg',
 '35_1_0_20170116222743531.jpg.chip.jpg',
 '28_0_2_20170116191256645.jpg.chip.jpg',
 '40773413_1985-05-18_2013.jpg',
 '918149_1890-07-11_1943.jpg',
 '7218188_1967-05-04_2012.jpg',
 '81_0_1_20170117193731174.jpg.chip.jpg',
 '70_0_0_20170117160355895.jpg.chip.jpg',
 '6215810_1973-11-27_2008.jpg',
 '18_1_0_20170109213856511.jpg.chip.jpg',
 '55_0_0_20170120223706509.jpg.chip.jpg',
 '003723.jpg',
 '3569434_1980-10-01_2012.jpg',
 '25_1_0_20170117193539276.jpg.chip.jpg',
 '19216583_1937-05-10_1964.jpg',
 '12636288_1975-11-30_2008.jpg',
 '41347756_1991-03-10_2012.jpg',
 '12417728_1949-02-02_1980.jpg',
 '37684007_1945-05-07_2012.jpg',
 '004260.jpg',
 '16_1_0_20170109213608540.jpg.chip.jpg',
 '44695408_1911-02-24_1991.jpg',
 '1639687_1967-07-14_1996.jpg',
 '19345924_1988-09-13_2008.jpg',
 '2062331_1976-06-18_2013.jpg',
 '3628240_1960-12-17_1987.jpg',
 '10529976_1930-05-20_1954.jpg',
 '27_0_1_20170117175753874.jpg.chip.jpg',
 '46_0_3_20170119204802431.jpg.chip.jpg',
 '41470713_1952-12-26_1974.jpg',
 '180733_1935-05-08_1969.jpg',
 '18900914_1985-01-02_2006.jpg',
 '22421796_1955-06-25_1983.jpg',
 '71_1_0_20170110140328613.jpg.chip.jpg',
 '2476333_1981-06-01_2010.jpg',
 '17_1_0_20170109212920120.jpg.chip.jpg',
 '1167372_1903-08-14_1952.jpg',
 '16555375_1988-11-12_2015.jpg',
 '2323881_1971-03-05_2007.jpg',
 '2_0_2_20161219141647272.jpg.chip.jpg',
 '16_0_0_20170110232332618.jpg.chip.jpg',
 '40826007_1996-05-31_2014.jpg',
 '27091124_1988-03-14_2011.jpg',
 '41_0_1_20170117173648607.jpg.chip.jpg',
 '41_0_4_20170104205417148.jpg.chip.jpg',
 '36242663_1992-04-13_2012.jpg',
 '32999146_1949-07-22_2011.jpg',
 '28509106_1927-11-12_2009.jpg',
 '24994_1956-09-23_2014.jpg',
 '20865712_1853-09-05_2004.jpg',
 '26_0_1_20170117154704178.jpg.chip.jpg',
 '258756_1951-07-31_1971.jpg',
 '378171_1941-12-01_2011.jpg',
 '32942324_1927-12-13_1951.jpg',
 '5767123_1984-12-20_2015.jpg',
 '23167046_1987-08-01_2015.jpg',
 '23560198_1992-06-20_2014.jpg',
 '2_1_3_20161220145026015.jpg.chip.jpg',
 '004750.jpg',
 '43730875_1991-07-25_2014.jpg',
 '36245423_1917-12-27_1985.jpg',
 '39440583_1906-10-16_1944.jpg',
 '001140.jpg',
 '35414147_1993-12-24_2012.jpg',
 '37763795_1946-06-22_2009.jpg',
 '5482429_1956-05-15_1978.jpg',
 '11189557_1943-10-02_2011.jpg',
 '6786417_1934-01-29_1972.jpg',
 '87637_1961-09-26_2013.jpg',
 '1088551_1934-10-01_1961.jpg',
 '560012_1954-12-31_2009.jpg',
 '36890678_1955-04-19_1970.jpg',
 '29_1_2_20170116173225104.jpg.chip.jpg',
 '32466540_1964-11-09_2013.jpg',
 '43663635_1988-05-12_2012.jpg',
 '3329342_1982-02-15_2008.jpg',
 '2057979_1959-02-05_2013.jpg',
 '9612190_1986-03-05_2009.jpg',
 '537999_1974-01-14_2005.jpg',
 '1_1_4_20161221193744550.jpg.chip.jpg',
 '34_1_2_20170116193448303.jpg.chip.jpg',
 '001413.jpg',
 '31_1_1_20170116155750618.jpg.chip.jpg',
 '4095083_1987-04-26_2015.jpg',
 '6511626_1984-11-25_2011.jpg',
 '26_0_1_20170113134739602.jpg.chip.jpg',
 '2_0_0_20161219162303390.jpg.chip.jpg',
 '432575_1979-07-03_2013.jpg',
 '505670_1941-10-25_1974.jpg',
 '7144873_1987-04-12_2013.jpg',
 '18249084_1985-06-19_2012.jpg',
 '23864942_1981-05-31_2010.jpg',
 '12501_1941-12-08_2008.jpg',
 '696094_1963-04-18_2012.jpg',
 '003933.jpg',
 '45131582_1932-07-01_1954.jpg',
 '2275438_1977-09-28_2005.jpg',
 '35724503_1948-09-19_1966.jpg',
 '12878785_1989-04-30_2011.jpg',
 '34_0_3_20170119165846359.jpg.chip.jpg',
 '1266534_1975-11-03_2011.jpg',
 '16354477_1959-07-15_2008.jpg',
 '001617.jpg',
 '28_1_0_20170113001234883.jpg.chip.jpg',
 '11696343_1989-03-03_2009.jpg',
 '46518299_1923-05-28_1947.jpg',
 '961110_1959-09-14_2002.jpg',
 '20128467_1937-04-23_2008.jpg',
 '24476862_1989-05-11_2015.jpg',
 '5948348_1987-09-17_2007.jpg',
 '26_0_0_20170117172201507.jpg.chip.jpg',
 '30_0_0_20170119145340944.jpg.chip.jpg',
 '1_0_2_20161219162852759.jpg.chip.jpg',
 '13283895_1976-01-10_2000.jpg',
 '1230336_1955-09-25_2007.jpg',
 '5302234_1939-10-16_1971.jpg',
 '41142422_1986-11-13_2014.jpg',
 '1_0_0_20170110213012671.jpg.chip.jpg',
 '5376225_1974-03-31_2009.jpg',
 '31730021_1984-11-09_2011.jpg',
 '27_1_1_20170117120629605.jpg.chip.jpg',
 '21633828_1984-12-05_2013.jpg',
 '35_0_3_20170119183157029.jpg.chip.jpg',
 '24_1_0_20170117150848786.jpg.chip.jpg',
 '3970134_1987-03-17_1980.jpg',
 '3078676_1944-01-29_2008.jpg',
 '11210768_1922-02-13_1948.jpg',
 '22316377_1989-06-01_2012.jpg',
 '37_0_1_20170116214356844.jpg.chip.jpg',
 '1145996_1948-07-22_2007.jpg',
 '26765106_1986-01-08_2014.jpg',
 '65_0_2_20170117000746182.jpg.chip.jpg',
 '36753693_1936-11-26_1965.jpg',
 '41707776_1991-10-11_2014.jpg',
 '4148260_1989-05-17_2010.jpg',
 '35_0_0_20170117134955604.jpg.chip.jpg',
 '1254277_1984-04-18_2010.jpg',
 '22892544_1923-11-01_2009.jpg',
 '2058760_1915-12-02_2012.jpg',
 '46471365_1935-10-08_1960.jpg',
 '29_1_0_20170111182452813.jpg.chip.jpg',
 '70_1_3_20170119150943854.jpg.chip.jpg',
 '25_0_4_20170117151414412.jpg.chip.jpg',
 '21741673_1995-03-01_2009.jpg',
 '148878_1965-10-05_2005.jpg',
 '13392046_1929-06-05_2002.jpg',
 '28682121_1988-04-28_2011.jpg',
 '33305711_1987-07-12_2012.jpg',
 '25_1_3_20170117152019467.jpg.chip.jpg',
 '41388906_1951-04-03_1974.jpg',
 '36529968_1979-10-25_2011.jpg',
 '45522223_1902-05-19_1944.jpg',
 '15699247_1942-07-07_1978.jpg',
 '70_0_3_20170117191256001.jpg.chip.jpg',
 '171517_1939-04-21_2004.jpg',
 '49_1_0_20170104235723082.jpg.chip.jpg',
 '45_0_4_20170117202958949.jpg.chip.jpg',
 '36348127_1990-03-23_2012.jpg',
 '26_0_2_20170116181346232.jpg.chip.jpg',
 '43223_1941-12-31_2006.jpg',
 '45376382_1996-10-30_2015.jpg',
 '7999955_1950-11-20_2011.jpg',
 '8978138_1915-01-22_1975.jpg',
 '55_0_0_20170117191627092.jpg.chip.jpg',
 '10123489_1984-01-26_2009.jpg',
 '890_1981-06-07_2009.jpg',
 '22104041_1987-05-12_2011.jpg',
 '3964439_1987-01-21_2009.jpg',
 '4054196_1974-01-04_2005.jpg',
 '5562149_1979-03-06_2006.jpg',
 '600233_1961-06-27_2013.jpg',
 '17145298_1984-11-18_2011.jpg',
 '2102231_1947-04-18_2008.jpg',
 '31_1_0_20170117133121639.jpg.chip.jpg',
 '1631693_1913-10-27_2009.jpg',
 '670462_1984-09-07_2011.jpg',
 '11253737_1918-08-11_1947.jpg',
 '30_0_1_20170117202813031.jpg.chip.jpg',
 '29485869_1990-06-07_2012.jpg',
 '29841300_1985-04-25_1980.jpg',
 '6944155_1971-06-05_2012.jpg',
 '41162605_1990-09-09_1990.jpg',
 '5771217_1989-01-17_2013.jpg',
 '3280564_1981-11-26_2014.jpg',
 '34123747_1966-03-12_2011.jpg',
 '001552.jpg',
 '23_0_3_20170119164205782.jpg.chip.jpg',
 '37104937_1995-02-13_2013.jpg',
 '58_0_3_20170119195320690.jpg.chip.jpg',
 '2697684_1916-01-19_2011.jpg',
 '7705333_1984-09-29_2009.jpg',
 '25709268_1943-03-10_2015.jpg',
 '26_0_3_20170119162753492.jpg.chip.jpg',
 '2108919_1973-10-26_2014.jpg',
 '4262860_1926-11-09_2010.jpg',
 '1917633_1964-04-04_1973.jpg',
 '26_1_3_20170117144854965.jpg.chip.jpg',
 '2068375_1964-08-25_2014.jpg',
 '12_1_4_20170109214232071.jpg.chip.jpg',
 '41490857_1964-07-01_1986.jpg',
 '33701731_1955-08-16_2002.jpg',
 '1967356_1968-09-02_2007.jpg',
 '55_0_0_20170116193520434.jpg.chip.jpg',
 '1180042_1930-04-03_1961.jpg',
 '1258483_1981-05-04_2008.jpg',
 '27420958_1990-03-29_2015.jpg',
 '26531370_1963-10-06_2012.jpg',
 '1783180_2000-10-31_2011.jpg',
 '35_0_1_20170113141808954.jpg.chip.jpg',
 '37_1_0_20170117140111378.jpg.chip.jpg',
 '31305274_1957-06-02_2003.jpg',
 '21005625_1918-01-13_1945.jpg',
 '6743730_1921-03-28_1984.jpg',
 '8920426_1983-05-31_2011.jpg',
 '43853176_1989-09-12_2013.jpg',
 '38_1_0_20170117154128620.jpg.chip.jpg',
 '16_1_4_20170103214347949.jpg.chip.jpg',
 '88_0_2_20170112224251748.jpg.chip.jpg',
 '26970398_1968-01-02_2011.jpg',
 '29_0_0_20170116192401102.jpg.chip.jpg',
 '1050883_1942-03-16_2002.jpg',
 '3806612_1950-10-08_1966.jpg',
 '46725990_1923-08-01_1948.jpg',
 '177180_1899-11-22_1947.jpg',
 '26_1_3_20170119193116354.jpg.chip.jpg',
 '003313.jpg',
 '4098246_1978-05-31_2005.jpg',
 '006636.jpg',
 '5977658_1935-03-19_1964.jpg',
 '26457252_1980-11-08_2013.jpg',
 '26_0_2_20170104022626733.jpg.chip.jpg',
 '28_1_2_20170116162435106.jpg.chip.jpg',
 '45054492_1979-12-30_2014.jpg',
 '34904258_1977-07-10_2014.jpg',
 '30974716_1986-10-12_2014.jpg',
 '24720665_1989-04-28_2015.jpg',
 '38385843_1991-01-12_2013.jpg',
 '65_1_0_20170110143236073.jpg.chip.jpg',
 '23253790_1894-12-05_2008.jpg',
 '1134432_1993-01-26_2011.jpg',
 '10110600_1985-09-17_2012.jpg',
 '32_1_0_20170108224936101.jpg.chip.jpg',
 '39794722_1922-09-30_1958.jpg',
 '1267629_1947-10-04_2010.jpg',
 '34411645_1985-04-23_2011.jpg',
 '20459229_1973-05-11_2011.jpg',
 '27_1_0_20170116220503099.jpg.chip.jpg',
 '39277041_1949-06-29_1995.jpg',
 '27_0_0_20170116232646294.jpg.chip.jpg',
 '82471_1961-03-16_2011.jpg',
 '46672958_1956-04-23_1978.jpg',
 '13184655_1990-06-21_2010.jpg',
 '233512_1972-07-07_2010.jpg',
 '001508.jpg',
 '35847193_1989-09-21_2014.jpg',
 '9976273_1960-08-15_1988.jpg',
 '32384638_1990-07-12_2009.jpg',
 '2121079_1981-08-26_2009.jpg',
 '10_0_0_20170110220654150.jpg.chip.jpg',
 '9838372_1906-09-01_2008.jpg',
 '284313_1943-04-24_2008.jpg',
 '28910941_1959-05-01_2008.jpg',
 '004375.jpg',
 '2404184_1926-04-17_1946.jpg',
 '31_0_1_20170116010634833.jpg.chip.jpg',
 '40081785_1966-10-27_1956.jpg',
 '24_0_1_20170113150729383.jpg.chip.jpg',
 '12_1_4_20161223230033380.jpg.chip.jpg',
 '8_1_0_20170109202926726.jpg.chip.jpg',
 '13073532_1934-08-09_2012.jpg',
 '000668.jpg',
 '7300200_1949-06-10_1940.jpg',
 '004515.jpg',
 '26_1_2_20170104022829221.jpg.chip.jpg',
 '8171044_1984-09-03_2009.jpg',
 '26_0_0_20170113210127395.jpg.chip.jpg',
 '473342_1957-06-14_2014.jpg',
 '22748865_1981-02-09_2004.jpg',
 '12677067_1979-01-24_2010.jpg',
 '1_1_0_20170109191129918.jpg.chip.jpg',
 '4415323_1983-05-24_2013.jpg',
 '50_0_3_20170119205556344.jpg.chip.jpg',
 '34827214_1976-02-16_1970.jpg',
 '26_1_0_20170117173613245.jpg.chip.jpg',
 '31436814_1983-01-29_2012.jpg',
 '36475873_1988-02-09_2012.jpg',
 '67749_1931-11-06_1970.jpg',
 '50_0_0_20170117135317385.jpg.chip.jpg',
 '004366.jpg',
 '9104418_1899-02-28_1942.jpg',
 '006172.jpg',
 '31028436_1979-11-02_2013.jpg',
 '2491752_1971-10-25_2004.jpg',
 '35061796_1964-12-07_2011.jpg',
 '39369330_1984-01-09_2008.jpg',
 '1_0_4_20161221201454961.jpg.chip.jpg',
 '44065241_1941-10-18_1966.jpg',
 '6_1_0_20170103212206588.jpg.chip.jpg',
 '21_1_3_20170104231613281.jpg.chip.jpg',
 '33_1_0_20170103181050000.jpg.chip.jpg',
 '45491923_1919-02-04_1946.jpg',
 '001847.jpg',
 '000171.jpg',
 '181232_1962-12-09_2007.jpg',
 '250278_1965-12-30_2006.jpg',
 '3984297_1956-05-12_2013.jpg',
 '26196169_1984-03-20_2010.jpg',
 '25660936_1963-06-06_2014.jpg',
 '38613357_1921-07-10_1961.jpg',
 '2821311_1947-12-30_1979.jpg',
 '32438826_1992-05-18_2013.jpg',
 '15422496_1988-04-28_2009.jpg',
 '24597672_1987-11-19_2010.jpg',
 '004507.jpg',
 '332134_1957-10-04_2009.jpg',
 '45569355_1995-10-29_2014.jpg',
 '26_1_0_20170117003252979.jpg.chip.jpg',
 '31712416_1980-06-19_2000.jpg',
 '23055210_1960-03-17_2013.jpg',
 '865964_1968-01-01_2009.jpg',
 '7623721_1978-03-01_2010.jpg',
 '38896201_2000-12-26_2014.jpg',
 '21699302_1978-09-02_2007.jpg',
 '72_0_0_20170111171747670.jpg.chip.jpg',
 '76_0_2_20170112204914106.jpg.chip.jpg',
 '39481001_1990-07-25_2012.jpg',
 '35_0_3_20170119212039049.jpg.chip.jpg',
 '40753749_1951-09-15_2008.jpg',
 '26_1_4_20170116220414012.jpg.chip.jpg',
 '2850239_1940-07-26_1974.jpg',
 '1944523_1976-01-09_2008.jpg',
 '30306491_1954-10-15_2014.jpg',
 '54_1_3_20170109142901150.jpg.chip.jpg',
 '6107450_1961-05-31_2010.jpg',
 '14410782_1925-02-08_1976.jpg',
 '36273685_1984-02-22_2013.jpg',
 '30_0_0_20170117181257163.jpg.chip.jpg',
 '218081_1918-02-01_1960.jpg',
 '7088375_1980-12-19_2006.jpg',
 '000679.jpg',
 '43399688_1994-08-03_2015.jpg',
 '3555882_1899-05-20_1940.jpg',
 '41365854_1980-06-19_2004.jpg',
 '16536131_1949-11-09_1980.jpg',
 '640889_1969-09-09_2008.jpg',
 '163892_1958-10-20_2012.jpg',
 '18888856_1989-03-19_2012.jpg',
 '31248425_1961-11-26_1990.jpg',
 '001744.jpg',
 '25918808_1982-06-22_2012.jpg',
 '529315_1906-01-14_1960.jpg',
 '28586654_1967-05-06_2007.jpg',
 '5468856_1976-04-07_2009.jpg',
 '26_1_0_20170117175317465.jpg.chip.jpg',
 '26_1_1_20170116231924481.jpg.chip.jpg',
 '34666103_1974-04-11_1996.jpg',
 '23_1_4_20170103233619053.jpg.chip.jpg',
 '26_1_3_20170119193148570.jpg.chip.jpg',
 '552779_1971-11-03_2006.jpg',
 '25_1_0_20170103180657288.jpg.chip.jpg',
 '161577_1980-06-08_2011.jpg',
 '26_1_1_20170116160509916.jpg.chip.jpg',
 '004715.jpg',
 '1925799_1967-01-30_2010.jpg',
 '207774_1925-06-09_1966.jpg',
 '23890218_1982-03-04_2009.jpg',
 '3313310_1965-04-28_2009.jpg',
 '4_1_3_20161219224933671.jpg.chip.jpg',
 '24179114_1938-10-20_2008.jpg',
 '571810_1956-09-17_2000.jpg',
 '651347_1955-04-06_2015.jpg',
 '005314.jpg',
 '15552802_1961-06-09_2009.jpg',
 '44108810_1926-06-28_1947.jpg',
 '34957545_1928-03-20_2013.jpg',
 '23877707_1926-07-23_1945.jpg',
 '327664_1951-01-12_1994.jpg',
 '36_0_0_20170113184113374.jpg.chip.jpg',
 '56_1_0_20170113182536919.jpg.chip.jpg',
 '8808513_1960-01-10_1983.jpg',
 '31621448_1991-04-04_2011.jpg',
 '3149083_1961-03-01_1990.jpg',
 '731994_1973-03-19_2007.jpg',
 '647373_1956-07-15_2010.jpg',
 '45396098_1976-01-10_2015.jpg',
 '1643259_1923-10-13_1961.jpg',
 '52_0_0_20170111201135454.jpg.chip.jpg',
 '195152_1931-11-26_2003.jpg',
 '23381415_1986-11-14_2011.jpg',
 '44119711_1819-11-17_2004.jpg',
 '252442_1896-08-15_1947.jpg',
 '1_0_4_20161219195147803.jpg.chip.jpg',
 '2377004_1972-01-30_2005.jpg',
 '32_1_0_20170116191618394.jpg.chip.jpg',
 '8087946_1944-04-21_1966.jpg',
 '13550616_1977-12-19_2010.jpg',
 '9322645_1971-08-26_2007.jpg',
 '10503610_1949-08-01_2005.jpg',
 '2385757_1961-05-08_2003.jpg',
 '2861958_1967-03-30_2011.jpg',
 '33983_1951-12-20_2000.jpg',
 '28_0_3_20170119194929915.jpg.chip.jpg',
 '524636_1961-04-08_2012.jpg',
 '30_1_0_20170117103601281.jpg.chip.jpg',
 '34876346_1946-05-13_2010.jpg',
 '34363325_1974-01-20_2001.jpg',
 '80_0_0_20170117175301125.jpg.chip.jpg',
 '28_0_0_20170116174356539.jpg.chip.jpg',
 '2_0_2_20161219202702788.jpg.chip.jpg',
 '22522134_1977-06-22_2002.jpg',
 '959464_1977-04-05_2004.jpg',
 '20597806_1980-10-04_2012.jpg',
 '28_0_0_20170116212102377.jpg.chip.jpg',
 '2134356_1966-03-14_2007.jpg',
 '36_0_3_20170119195506419.jpg.chip.jpg',
 '26_1_0_20170116164040161.jpg.chip.jpg',
 '5_0_4_20170103210509634.jpg.chip.jpg',
 '45681223_1993-06-25_2014.jpg',
 '43479903_1988-07-21_2014.jpg',
 '67030_1909-06-20_1940.jpg',
 '2857671_1920-02-14_1947.jpg',
 '36787370_1977-04-05_2010.jpg',
 '12167720_1984-09-30_2014.jpg',
 '10970703_1984-05-21_2011.jpg',
 '35022161_1984-11-13_2011.jpg',
 '45711961_1986-08-14_2008.jpg',
 '47_0_0_20170103183442514.jpg.chip.jpg',
 '3250661_1986-01-31_2009.jpg',
 '1896252_1942-09-02_2008.jpg',
 '5737964_1980-07-16_2007.jpg',
 '45277231_1926-02-26_1973.jpg',
 '40380141_1995-11-27_2013.jpg',
 '343294_1964-01-27_2001.jpg',
 '12754_1899-07-07_1946.jpg',
 '25648564_1936-12-07_1971.jpg',
 '7463952_1982-12-30_2013.jpg',
 '471630_1898-11-22_1978.jpg',
 '1148639_1963-12-26_2009.jpg',
 '40184108_1993-05-29_2013.jpg',
 '92331_1955-08-10_2008.jpg',
 '10016225_1988-02-03_2012.jpg',
 '40_0_2_20170116191354067.jpg.chip.jpg',
 '11700461_1983-10-11_2007.jpg',
 '32908796_1985-06-20_2013.jpg',
 '24_1_0_20170117191400699.jpg.chip.jpg',
 '24064016_1989-05-07_2008.jpg',
 '25211052_1991-07-08_2015.jpg',
 '23315533_1988-01-27_2011.jpg',
 '33999923_1993-04-30_2012.jpg',
 '79_1_0_20170120225803298.jpg.chip.jpg',
 '25615152_1987-11-11_2009.jpg',
 '11973758_1991-02-27_2008.jpg',
 '1246851_1974-08-09_2014.jpg',
 '26_0_2_20170116182347867.jpg.chip.jpg',
 '1235669_1976-09-11_2007.jpg',
 '2693444_1972-08-09_1965.jpg',
 '27_0_4_20170103235757172.jpg.chip.jpg',
 '30387921_1975-02-01_2010.jpg',
 '3331742_1969-03-26_2013.jpg',
 '17021305_1883-02-28_1967.jpg',
 '41691585_1995-09-25_2015.jpg',
 '3101416_1973-03-17_2009.jpg',
 '53_1_0_20170109220736654.jpg.chip.jpg',
 '34027685_1972-07-22_2005.jpg',
 '7514390_1977-05-25_2014.jpg',
 '3348502_1984-09-18_2007.jpg',
 '11021240_1887-08-31_1954.jpg',
 '1090974_1971-03-12_1988.jpg',
 '35_1_1_20170113012807626.jpg.chip.jpg',
 '913724_1979-04-03_2008.jpg',
 '34557038_1974-07-08_2011.jpg',
 '006995.jpg',
 '37_1_1_20170116232758932.jpg.chip.jpg',
 '5929080_1943-07-02_1997.jpg',
 '6171937_1981-05-29_2007.jpg',
 '2251257_1931-03-23_1964.jpg',
 '2776174_1969-02-20_2014.jpg',
 '65_1_0_20170103184138483.jpg.chip.jpg',
 '7856897_1986-05-23_2012.jpg',
 '46182062_1969-04-29_2013.jpg',
 '38128011_1979-08-14_2002.jpg',
 '1776092_1959-01-31_2007.jpg',
 '4647550_1949-10-31_2012.jpg',
 '2092862_1956-04-09_1984.jpg',
 '2201_1922-06-19_1955.jpg',
 '68_0_0_20170117172938504.jpg.chip.jpg',
 '33369667_1989-01-28_2012.jpg',
 '8118857_1953-06-11_1977.jpg',
 '28968145_1985-04-03_2007.jpg',
 '1768989_1953-07-21_2008.jpg',
 '849454_1955-04-17_2013.jpg',
 '26908645_1973-03-22_2015.jpg',
 '18109220_1987-08-04_2015.jpg',
 '8286385_1922-11-01_1953.jpg',
 '1405558_1968-03-14_2012.jpg',
 '26_1_1_20170112212042603.jpg.chip.jpg',
 '21222403_1983-09-29_2012.jpg',
 '593572_1960-09-04_2007.jpg',
 '32_1_1_20170116164948959.jpg.chip.jpg',
 '3776811_1938-01-05_1990.jpg',
 '42737444_1674-04-22_2014.jpg',
 '18_0_0_20170117140126568.jpg.chip.jpg',
 '20681612_1925-11-04_1947.jpg',
 '30_0_2_20170117141818577.jpg.chip.jpg',
 '004472.jpg',
 '22_1_1_20170112233701320.jpg.chip.jpg',
 '4334396_1972-09-07_1993.jpg',
 '27599355_1985-04-07_2007.jpg',
 '60_0_0_20170117160634375.jpg.chip.jpg',
 '6042299_1975-01-05_1970.jpg',
 '3214260_1968-09-12_2012.jpg',
 '15208001_1981-02-08_2012.jpg',
 '27_0_0_20170117175924156.jpg.chip.jpg',
 '23542494_1976-03-21_2012.jpg',
 '37867537_1965-02-11_1984.jpg',
 '37484443_1991-03-17_2013.jpg',
 '1_1_3_20161219224457585.jpg.chip.jpg',
 '18284846_1989-09-02_2010.jpg',
 '2456295_1969-10-14_2008.jpg',
 '20_1_0_20170103201733831.jpg.chip.jpg',
 '41334185_1991-06-24_2014.jpg',
 '17814123_1930-08-16_1974.jpg',
 '36873309_1959-12-01_2010.jpg',
 '65_1_0_20170110141141881.jpg.chip.jpg',
 '8484508_1981-12-31_2010.jpg',
 '26_0_2_20170119184102087.jpg.chip.jpg',
 '54_0_0_20170105163714131.jpg.chip.jpg',
 '31_0_1_20170113175408636.jpg.chip.jpg',
 '18676025_1969-06-11_2008.jpg',
 '26_1_4_20170117154137164.jpg.chip.jpg',
 '001075.jpg',
 '28_0_0_20170119194754322.jpg.chip.jpg',
 '73_0_0_20170120230233019.jpg.chip.jpg',
 '42412485_1931-09-03_1966.jpg',
 '26_1_4_20170103230455729.jpg.chip.jpg',
 '14013061_1977-11-07_2006.jpg',
 '72_0_2_20170112224532491.jpg.chip.jpg',
 '36_0_3_20170119194656666.jpg.chip.jpg',
 '1_0_2_20161219201442357.jpg.chip.jpg',
 '7285573_1929-05-07_1962.jpg',
 '001164.jpg',
 '6015495_1958-09-27_1950.jpg',
 '14996024_1982-09-16_2013.jpg',
 '2195339_1953-11-23_2008.jpg',
 '38480304_1911-04-12_2011.jpg',
 '42219298_1990-02-14_2014.jpg',
 '60_0_3_20170119205525232.jpg.chip.jpg',
 '2161453_1950-02-03_1973.jpg',
 '37464471_1957-05-22_1980.jpg',
 '38397939_1937-06-03_1960.jpg',
 '33319744_1992-04-26_1990.jpg',
 '3428686_1965-06-18_2015.jpg',
 '38_0_1_20170116003732637.jpg.chip.jpg',
 '2178459_1985-01-14_2008.jpg',
 '13_1_0_20170109204325267.jpg.chip.jpg',
 '20545506_1976-07-08_2009.jpg',
 '70_0_0_20170117173727535.jpg.chip.jpg',
 '005296.jpg',
 '26_1_2_20170116175903368.jpg.chip.jpg',
 '26_1_4_20170117201351845.jpg.chip.jpg',
 '24232963_1987-10-02_2012.jpg',
 '4_1_1_20170117202024231.jpg.chip.jpg',
 '5388344_1978-06-22_2012.jpg',
 '1497643_1916-08-14_1954.jpg',
 '004802.jpg',
 '37791562_1991-02-15_2014.jpg',
 '24_1_3_20170104223019607.jpg.chip.jpg',
 '11787967_1967-04-02_1960.jpg',
 '10_0_0_20170110225444491.jpg.chip.jpg',
 '6731123_1953-07-05_2011.jpg',
 '26897013_1991-06-25_2010.jpg',
 '002183.jpg',
 '42345_1900-01-27_1955.jpg',
 '863211_1930-06-01_1971.jpg',
 '22698827_1985-12-16_2013.jpg',
 '10838173_1990-02-01_2012.jpg',
 '8224016_1973-03-15_1970.jpg',
 '3660144_1972-04-13_1970.jpg',
 '42842300_1994-10-18_2014.jpg',
 '1515927_1980-09-26_2005.jpg',
 '307526_1964-09-08_2013.jpg',
 '6407782_1977-12-20_2007.jpg',
 '35937539_1992-08-08_2012.jpg',
 '33124994_1992-01-07_2011.jpg',
 '53_1_3_20170109141903857.jpg.chip.jpg',
 '40_0_2_20170116181214637.jpg.chip.jpg',
 '23_1_0_20170117144945297.jpg.chip.jpg',
 '003552.jpg',
 '23_1_3_20170117144858514.jpg.chip.jpg',
 '44059509_1928-01-08_1954.jpg',
 '24310807_1945-01-18_2009.jpg',
 '38634484_1993-07-30_2012.jpg',
 '8980506_1979-07-14_2009.jpg',
 '1042604_1956-06-04_2010.jpg',
 '64_0_0_20170104185632214.jpg.chip.jpg',
 '32_1_2_20170116191623170.jpg.chip.jpg',
 ...]
df_wiki_utk_appa.hist()
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7fcf163f9c88>]], dtype=object)
df_wiki_utk_appa['age'].value_counts()
26     2632
24     1593
28     1575
25     1573
30     1382
       ... 
98        2
97        2
101       1
111       1
103       1
Name: age, Length: 107, dtype: int64
df_wiki_utk_appa = df_wiki_utk_appa[df_wiki_utk_appa['age'] <= 100]
df_wiki_utk_appa = df_wiki_utk_appa[df_wiki_utk_appa['age'] > 0]
df_wiki_utk_appa['age'].min(), df_wiki_utk_appa['age'].max()
(1, 100)
df_wiki_utk_appa['age'] = df_wiki_utk_appa['age'].astype(int)
path_wiki_utk_appa = Path('../working/wiki_utk/')

np.random.seed(42)

data_wiki_small_src = (ImageList.from_df(df_wiki_utk_appa, path_wiki_utk_appa, cols=['full_path'], folder='.')
                   .split_by_rand_pct(0.2, seed=42)
                   .label_from_df(label_cls=FloatList))
                   
# data_wiki_big = ImageList.from_df(df_wiki_utk_appa, path_wiki_utk_appa, cols=['full_path'], folder='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_wiki_small = (data_wiki_small_src.transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128)
                   .databunch(bs=64,num_workers=0).normalize(imagenet_stats))
data_wiki_small.show_batch(4, figsize=(12,12))

Fastai Age Model

class AgeModel(nn.Module):
    def __init__(self):
        super().__init__()
        layers = list(models.resnet34(pretrained=True).children())[:-2]
        layers += [AdaptiveConcatPool2d(), Flatten()]
        #layers += [nn.BatchNorm1d(4096, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
        #layers += [nn.Dropout(p=0.60)]
        #layers += [nn.Linear(4096, 1024, bias=True), nn.ReLU(inplace=True)]
        #layers += [nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
        #layers += [nn.Dropout(p=0.60)]
        #layers += [nn.Linear(2048, 1024, bias=True), nn.ReLU(inplace=True)]
        #layers += [nn.BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
        #layers += [nn.Dropout(p=0.75)]
        #layers += [nn.Linear(1024, 256, bias=True), nn.ReLU(inplace=True)]
        #layers += [nn.BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
        #layers += [nn.Dropout(p=0.50)]
        #layers += [nn.Linear(512,256 , bias=True), nn.ReLU(inplace=True)]
        layers += [nn.BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
        layers += [nn.Dropout(p=0.50)]
        layers += [nn.Linear(1024, 512, bias=True), nn.ReLU(inplace=True)]
        layers += [nn.BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
        layers += [nn.Dropout(p=0.50)]
        layers += [nn.Linear(512, 16, bias=True), nn.ReLU(inplace=True)]
        layers += [nn.Linear(16,1)]
        self.agemodel = nn.Sequential(*layers)
    def forward(self, x):
        return self.agemodel(x).squeeze(-1)
          # could add 116*torch.sigmoid
model = AgeModel()
Downloading: "https://download.pytorch.org/models/resnet34-333f7ec4.pth" to /tmp/.cache/torch/checkpoints/resnet34-333f7ec4.pth
100%|██████████| 83.3M/83.3M [00:00<00:00, 92.9MB/s]
learn = Learner(data_wiki_small, model, model_dir = "/temp/model/", opt_func=opt_func, bn_wd=False, metrics=root_mean_squared_error,
               callback_fns=[ShowGraph]).mixup(stack_y=False, alpha=0.2)

learn.loss_func = L1LossFlat()
learn.split([model.agemodel[4],model.agemodel[6],model.agemodel[8]])
Learner(data=ImageDataBunch;

Train: LabelList (34909 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;

Valid: LabelList (8727 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;

Test: None, model=AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): ReLU(inplace=True)
  (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): ReLU(inplace=True)
  (13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): ReLU(inplace=True)
  (18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): ReLU(inplace=True)
  (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (14): ReLU(inplace=True)
  (15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (19): ReLU(inplace=True)
  (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (41): ReLU(inplace=True)
  (42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (46): ReLU(inplace=True)
  (47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): AdaptiveAvgPool2d(output_size=1)
  (1): AdaptiveMaxPool2d(output_size=1)
  (2): Flatten()
  (3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (4): Dropout(p=0.5, inplace=False)
  (5): Linear(in_features=1024, out_features=512, bias=True)
  (6): ReLU(inplace=True)
  (7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Dropout(p=0.5, inplace=False)
  (9): Linear(in_features=512, out_features=16, bias=True)
  (10): ReLU(inplace=True)
  (11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=False)
#learn = cnn_learner(data, resnet50, pretrained = True, model_dir = "/temp/model/",callback_fns=[ShowGraph])
#learn.loss_func = L1LossFlat()
import fastai
learn.freeze_to(-1)
learn.lr_find()
learn.recorder.plot(suggestion = True)
LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 3.98E-02
Min loss divided by 10: 9.12E-02
lr = 2e-2
learn.fit_one_cycle(5, max_lr=slice(lr), wd=(1e-6, 1e-4, 1e-2, 1e-1), pct_start=0.5, callbacks=[SaveModelCallback(learn)])
epoch train_loss valid_loss root_mean_squared_error time
0 9.605057 9.252867 13.258863 07:27
1 8.392913 7.800601 11.307045 07:26
2 7.934024 8.536113 12.432997 07:25
3 7.214382 6.707546 9.728629 07:25
4 7.115435 6.620564 9.895959 07:26
Better model found at epoch 0 with valid_loss value: 9.252866744995117.
Better model found at epoch 1 with valid_loss value: 7.800601005554199.
Better model found at epoch 3 with valid_loss value: 6.707545757293701.
Better model found at epoch 4 with valid_loss value: 6.620563507080078.
learn.save('first_head_resnet34')
learn.load('first_head_resnet34')
Learner(data=ImageDataBunch;

Train: LabelList (34909 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;

Valid: LabelList (8727 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;

Test: None, model=AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): ReLU(inplace=True)
  (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): ReLU(inplace=True)
  (13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): ReLU(inplace=True)
  (18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): ReLU(inplace=True)
  (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (14): ReLU(inplace=True)
  (15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (19): ReLU(inplace=True)
  (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (41): ReLU(inplace=True)
  (42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (46): ReLU(inplace=True)
  (47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): AdaptiveAvgPool2d(output_size=1)
  (1): AdaptiveMaxPool2d(output_size=1)
  (2): Flatten()
  (3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (4): Dropout(p=0.5, inplace=False)
  (5): Linear(in_features=1024, out_features=512, bias=True)
  (6): ReLU(inplace=True)
  (7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Dropout(p=0.5, inplace=False)
  (9): Linear(in_features=512, out_features=16, bias=True)
  (10): ReLU(inplace=True)
  (11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)
learn.unfreeze()
learn.lr_find()
learn.recorder.plot(suggestion = True)
LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 1.91E-06
Min loss divided by 10: 6.92E-07
learn.fit_one_cycle(5, max_lr=slice(1e-6, lr/5), wd=(1e-6, 1e-4, 1e-2, 1e-1), 
                    callbacks=[SaveModelCallback(learn)], pct_start=0.5)
epoch train_loss valid_loss root_mean_squared_error time
0 7.085314 7.008016 10.614280 07:32
1 7.218247 6.842098 9.879888 07:31
2 6.800269 6.805061 9.620695 07:30
3 6.229340 5.717019 8.594589 07:30
4 6.011308 5.505085 8.282001 07:31
Better model found at epoch 0 with valid_loss value: 7.008016109466553.
Better model found at epoch 1 with valid_loss value: 6.842098236083984.
Better model found at epoch 2 with valid_loss value: 6.805061340332031.
Better model found at epoch 3 with valid_loss value: 5.7170186042785645.
Better model found at epoch 4 with valid_loss value: 5.505085468292236.
x,y = next(iter(learn.data.train_dl))
x.shape
torch.Size([64, 3, 128, 128])
learn.save('first_body_resnet34')
learn.load('first_body_resnet34')
Learner(data=ImageDataBunch;

Train: LabelList (34909 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;

Valid: LabelList (8727 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;

Test: None, model=AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): ReLU(inplace=True)
  (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): ReLU(inplace=True)
  (13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): ReLU(inplace=True)
  (18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): ReLU(inplace=True)
  (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (14): ReLU(inplace=True)
  (15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (19): ReLU(inplace=True)
  (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (41): ReLU(inplace=True)
  (42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (46): ReLU(inplace=True)
  (47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): AdaptiveAvgPool2d(output_size=1)
  (1): AdaptiveMaxPool2d(output_size=1)
  (2): Flatten()
  (3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (4): Dropout(p=0.5, inplace=False)
  (5): Linear(in_features=1024, out_features=512, bias=True)
  (6): ReLU(inplace=True)
  (7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Dropout(p=0.5, inplace=False)
  (9): Linear(in_features=512, out_features=16, bias=True)
  (10): ReLU(inplace=True)
  (11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)
learn.show_results()
img = open_image('../input/picture4/PM_Modi_2015.jpg')
img
x = learn.predict(img)
x
(FloatItem 61.401455, tensor(61.4015), tensor(61.4015))

Image Resizing

data_wiki_big = (data_wiki_small_src.transform(tfms, size=256)
        .databunch(num_workers=0).normalize(imagenet_stats))

learn.data = data_wiki_big
data_wiki_big.train_ds[0][0].shape
torch.Size([3, 256, 256])
learn.freeze_to(-1)
learn.lr_find()
learn.recorder.plot(suggestion = True)
LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 9.12E-07
Min loss divided by 10: 5.75E-03
lr = 5e-4
learn.fit_one_cycle(5, max_lr=slice(lr), wd=(1e-6, 1e-4, 1e-2, 1e-1), pct_start=0.5, callbacks=[SaveModelCallback(learn)])
epoch train_loss valid_loss root_mean_squared_error time
0 7.837380 6.402583 9.601757 14:34
1 7.332215 6.078465 9.117067 14:26
2 7.223328 5.939247 8.911067 14:18
3 6.803191 5.813476 8.721518 14:18
4 7.061768 5.878111 8.838454 14:21
Better model found at epoch 0 with valid_loss value: 6.402583122253418.
Better model found at epoch 1 with valid_loss value: 6.078464984893799.
Better model found at epoch 2 with valid_loss value: 5.939246654510498.
Better model found at epoch 3 with valid_loss value: 5.813475608825684.
learn.recorder.plot_lr()
learn.recorder.plot_losses()
learn.save('second_head_resnet34')
learn.load('second_head_resnet34')
Learner(data=ImageDataBunch;

Train: LabelList (34909 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;

Valid: LabelList (8727 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;

Test: None, model=AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): ReLU(inplace=True)
  (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): ReLU(inplace=True)
  (13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): ReLU(inplace=True)
  (18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): ReLU(inplace=True)
  (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (14): ReLU(inplace=True)
  (15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (19): ReLU(inplace=True)
  (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (41): ReLU(inplace=True)
  (42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (46): ReLU(inplace=True)
  (47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): AdaptiveAvgPool2d(output_size=1)
  (1): AdaptiveMaxPool2d(output_size=1)
  (2): Flatten()
  (3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (4): Dropout(p=0.5, inplace=False)
  (5): Linear(in_features=1024, out_features=512, bias=True)
  (6): ReLU(inplace=True)
  (7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Dropout(p=0.5, inplace=False)
  (9): Linear(in_features=512, out_features=16, bias=True)
  (10): ReLU(inplace=True)
  (11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)
img = open_image('../input/picture4/PM_Modi_2015.jpg')
img
learn.predict(img)
(FloatItem 61.38315, tensor(61.3831), tensor(61.3831))
x,y = next(iter(learn.data.train_dl))
x.shape
torch.Size([64, 3, 256, 256])
learn.unfreeze()
learn.lr_find()
learn.recorder.plot(suggestion = True)
LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 9.12E-07
Min loss divided by 10: 3.02E-06
learn.fit_one_cycle(5, max_lr=slice(1e-6,1e-4), wd=(1e-6, 1e-4, 1e-2, 1e-1), callbacks=[SaveModelCallback(learn)], pct_start=0.5)
epoch train_loss valid_loss root_mean_squared_error time
0 6.700283 5.710420 8.648839 14:52
1 6.323674 5.421929 8.128598 15:00
2 6.052965 5.245904 7.984351 14:55
3 5.895456 5.199911 7.924792 14:54
4 5.728145 5.103240 7.743491 14:55
Better model found at epoch 0 with valid_loss value: 5.71042013168335.
Better model found at epoch 1 with valid_loss value: 5.421929359436035.
Better model found at epoch 2 with valid_loss value: 5.245903968811035.
Better model found at epoch 3 with valid_loss value: 5.199910640716553.
Better model found at epoch 4 with valid_loss value: 5.103240489959717.
learn.save('second_body_resnet34')
learn.load('second_body_resnet34')
Learner(data=ImageDataBunch;

Train: LabelList (34909 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;

Valid: LabelList (8727 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;

Test: None, model=AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): ReLU(inplace=True)
  (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): ReLU(inplace=True)
  (13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): ReLU(inplace=True)
  (18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): ReLU(inplace=True)
  (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (14): ReLU(inplace=True)
  (15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (19): ReLU(inplace=True)
  (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (41): ReLU(inplace=True)
  (42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (46): ReLU(inplace=True)
  (47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): AdaptiveAvgPool2d(output_size=1)
  (1): AdaptiveMaxPool2d(output_size=1)
  (2): Flatten()
  (3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (4): Dropout(p=0.5, inplace=False)
  (5): Linear(in_features=1024, out_features=512, bias=True)
  (6): ReLU(inplace=True)
  (7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Dropout(p=0.5, inplace=False)
  (9): Linear(in_features=512, out_features=16, bias=True)
  (10): ReLU(inplace=True)
  (11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)
learn.show_results()
img = open_image('../input/picture4/PM_Modi_2015.jpg')
img
x = learn.predict(img)
x
(FloatItem 63.15648, tensor(63.1565), tensor(63.1565))
learn.save('third_body_resnet34')
learn.load('third_body_resnet34')
Learner(data=ImageDataBunch;

Train: LabelList (34909 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;

Valid: LabelList (8727 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;

Test: None, model=AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): ReLU(inplace=True)
  (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): ReLU(inplace=True)
  (13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): ReLU(inplace=True)
  (18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): ReLU(inplace=True)
  (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (14): ReLU(inplace=True)
  (15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (19): ReLU(inplace=True)
  (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (41): ReLU(inplace=True)
  (42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (46): ReLU(inplace=True)
  (47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): AdaptiveAvgPool2d(output_size=1)
  (1): AdaptiveMaxPool2d(output_size=1)
  (2): Flatten()
  (3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (4): Dropout(p=0.5, inplace=False)
  (5): Linear(in_features=1024, out_features=512, bias=True)
  (6): ReLU(inplace=True)
  (7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Dropout(p=0.5, inplace=False)
  (9): Linear(in_features=512, out_features=16, bias=True)
  (10): ReLU(inplace=True)
  (11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)

Fastai Hooks

class SaveFeatures():
    features=None
    def __init__(self, m): 
        self.hook = m.register_forward_hook(self.hook_fn)
        self.features = None
    def hook_fn(self, module, input, output): 
        out = output.detach().cpu().numpy()
        if isinstance(self.features, type(None)):
            self.features = out
        else:
            self.features = np.row_stack((self.features, out))
    def remove(self): 
        self.hook.remove()
learn.model
AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
)
# Second last layer of the model
learn.model.agemodel[16]
Linear(in_features=512, out_features=16, bias=True)
len(data_wiki_big.train_ds.items), len(data_wiki_big.valid_ds.items)
(34909, 8727)
sf = SaveFeatures(learn.model.agemodel[16])
_= learn.get_preds(data_wiki_big.train_ds)
_= learn.get_preds(DatasetType.Valid)
len(sf.features)
43636
img_path = [str(x) for x in (list(data_wiki_big.train_ds.items) +list(data_wiki_big.valid_ds.items))]
label = [x for x in (list(data_wiki_big.train_ds.y.items) +list(data_wiki_big.valid_ds.y.items))]
label_id = [x for x in (list(data_wiki_big.train_ds.y.items) +list(data_wiki_big.valid_ds.y.items))]
df_new = pd.DataFrame({'img_path': img_path, 'label': label, 'label_id': label_id})
df_new
img_path label label_id
0 ../working/wiki_utk/./10002702_1960-11-09_2012... 52.0 52.0
1 ../working/wiki_utk/./10003541_1937-09-27_1971... 34.0 34.0
2 ../working/wiki_utk/./1000456_1933-06-12_1969.jpg 36.0 36.0
3 ../working/wiki_utk/./10004882_1987-05-16_2010... 23.0 23.0
4 ../working/wiki_utk/./1000781_1930-07-07_2008.jpg 78.0 78.0
... ... ... ...
43631 ../working/wiki_utk/./007148.jpg 20.0 20.0
43632 ../working/wiki_utk/./4948808_1983-11-16_2007.jpg 24.0 24.0
43633 ../working/wiki_utk/./6_1_4_20170119204208503.... 6.0 6.0
43634 ../working/wiki_utk/./26_0_4_20170117154446109... 26.0 26.0
43635 ../working/wiki_utk/./18_0_4_20170103201305015... 18.0 18.0

43636 rows × 3 columns

array = np.array(sf.features)
x=array.tolist()
df_new['img_repr'] = x
df_new
img_path label label_id img_repr
0 ../working/wiki_utk/./10002702_1960-11-09_2012... 52.0 52.0 [72.47952270507812, -8.392868041992188, -9.617...
1 ../working/wiki_utk/./10003541_1937-09-27_1971... 34.0 34.0 [72.01614379882812, -7.710638999938965, -8.909...
2 ../working/wiki_utk/./1000456_1933-06-12_1969.jpg 36.0 36.0 [69.15504455566406, -7.593712329864502, -8.700...
3 ../working/wiki_utk/./10004882_1987-05-16_2010... 23.0 23.0 [49.39116668701172, -7.189101696014404, -7.045...
4 ../working/wiki_utk/./1000781_1930-07-07_2008.jpg 78.0 78.0 [134.6729736328125, -16.601808547973633, -19.6...
... ... ... ... ...
43631 ../working/wiki_utk/./007148.jpg 20.0 20.0 [42.290016174316406, -7.002415657043457, -6.35...
43632 ../working/wiki_utk/./4948808_1983-11-16_2007.jpg 24.0 24.0 [28.6945743560791, -7.378058910369873, -5.9204...
43633 ../working/wiki_utk/./6_1_4_20170119204208503.... 6.0 6.0 [49.36155700683594, -6.653528213500977, -6.824...
43634 ../working/wiki_utk/./26_0_4_20170117154446109... 26.0 26.0 [49.14276885986328, -5.3559465408325195, -5.08...
43635 ../working/wiki_utk/./18_0_4_20170103201305015... 18.0 18.0 [21.474647521972656, -6.6602959632873535, -4.7...

43636 rows × 4 columns

df_new.shape
(43636, 4)

Annoy

from annoy import AnnoyIndex

f = len(df_new['img_repr'][0])
t = AnnoyIndex(f, metric='euclidean')
ntree = 1000

for i, vector in enumerate(df_new['img_repr']):
    t.add_item(i, vector)
_  = t.build(ntree)
import time
def get_similar_images_annoy(img_index, num):
    start = time.time()
    base_img_id, base_vector, base_label  = df_new.iloc[img_index, [0,3,1]]
    similar_img_ids = t.get_nns_by_item(img_index, num)
    end = time.time()
    print(f'{(end - start) * 1000} ms')
    return base_img_id, base_label, df_new.iloc[similar_img_ids]
num = 89
base_image, base_label, similar_images_df = get_similar_images_annoy(890, num)
70.15395164489746 ms
print(base_label)
open_image(base_image)
21.0
similar_images_df
img_path label label_id img_repr
890 ../working/wiki_utk/./950234_1941-03-06_1962.jpg 21.0 21.0 [44.051395416259766, -8.610692024230957, -7.61...
10285 ../working/wiki_utk/./1575279_1978-12-28_2014.jpg 36.0 36.0 [44.23878479003906, -8.540730476379395, -7.615...
40921 ../working/wiki_utk/./23465846_1989-07-10_2010... 21.0 21.0 [44.385948181152344, -8.653063774108887, -7.68...
11172 ../working/wiki_utk/./4182570_1978-06-26_2012.jpg 34.0 34.0 [44.501441955566406, -8.626776695251465, -7.64...
34427 ../working/wiki_utk/./007004.jpg 27.0 27.0 [44.107200622558594, -8.570219993591309, -7.66...
... ... ... ... ...
3505 ../working/wiki_utk/./25239834_1992-01-23_2015... 23.0 23.0 [44.192176818847656, -8.403630256652832, -7.45...
40902 ../working/wiki_utk/./1691972_1933-02-18_1968.jpg 35.0 35.0 [45.43599319458008, -8.536643981933594, -7.595...
12401 ../working/wiki_utk/./1190101_1979-11-13_2010.jpg 31.0 31.0 [45.04903793334961, -8.50577449798584, -7.5224...
40885 ../working/wiki_utk/./4094157_1990-03-01_2009.jpg 19.0 19.0 [43.83957290649414, -8.407843589782715, -7.513...
13491 ../working/wiki_utk/./589278_1897-01-23_1942.jpg 45.0 45.0 [45.358192443847656, -8.5722074508667, -7.6586...

89 rows × 4 columns

data_src = (ImageList.from_df(similar_images_df, path=".",cols=['img_path'], folder='.' )
        .split_none()
        .label_from_df(cols=['label'],label_cls=FloatList))
data = (data_src.transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256)
                   .databunch(bs=num,num_workers=0).normalize(imagenet_stats))
data.show_batch(2)